All the Images of an Outdoor Scene

The appearance of an outdoor scene depends on a variety of factors such as viewing geometry, scene structure and reflectance (BRDF or BTF), illumination (sun, moon, stars, street lamps), atmospheric condition (clear air, fog, rain) and weathering (or aging) of materials. Over time, these factors change, altering the way a scene appears. A large set of images is required to study the entire variability in scene appearance. In this paper, we present a database of high quality registered and calibrated images of a fixed outdoor scene captured every hour for over 5 months. The dataset covers a wide range of daylight and night illumination conditions, weather conditions and seasons. We describe in detail the image acquisition and sensor calibration procedures. The images are tagged with a variety of ground truth data such as weather and illumination conditions and actual scene depths. This database has potential implications for vision, graphics, image processing and atmospheric sciences and can be a testbed for many algorithms. We describe an example application image analysis in bad weather and show how this method can be evaluated using the images in the database. The database is available online at http://www.cs.columbia.edu/CAVE/. The data collection is ongoing and we plan to acquire images for one year. 1 Variability in Scene Appearance The appearance of a fixed scene depends on several factors the viewing geometry, illumination geometry and spectrum, scene structure and reflectance (BRDF or BTF) and the medium (say, atmosphere) in which the scene is immersed. The estimation of one or more of these appearance parameters from one or more images of the scene has been an important part of research in computer vision. Several researchers have focused on solving this inverse problem under specific conditions of illumination (constant or smoothly varying), scene structure (no discontinuities), BRDF (lambertian) and transparent media (pure air). Images captured to evaluate their methods adhere to the specific conditions. While understanding each of these specific cases is important, modeling scene appearance in the most general setting is ultimately the goal of a vision system. To model, develop and evaluate such a general vision system, it is critical to collect a comprehensive set of images that describes the complete variability in the appearance of a scene. Several research groups have collected images of a scene (for example, faces, textures, objects) under varying lighting conditions A. Heyden et al. (Eds.): ECCV 2002, LNCS 2352, pp. 148–162, 2002. c © Springer-Verlag Berlin Heidelberg 2002 All the Images of an Outdoor Scene 149 and/or viewpoints in controlled lab environments. The CMU-PIE database [1] has 40000 facial images under different poses, illumination directions and facial expressions. The FERET [2] database consists of 1196 images of faces with varying facial expressions. Similarly, the Yale Face database [3] has around 165 images taken under different lighting, pose and occlusion configurations. The SLAM database [4] provides a set of 1500 images of toy objects under different poses. The color constancy dataset collected by Funt et al. [5] provides a large set of images of objects (boxes, books and so on) acquired under different poses and with different illuminants (fluorescent, halogen, etc). The CURET database [6] provides a set of 12000 images of real world textures under 200 illumination and viewing configurations. It also provides an additional set of 14000 Bi-directional Texture Function (BTF) measurements of 61 real world surfaces. Several databases of images of outdoor scenes have also been collected. The “natural stimuli collection” [7] has around 4000 images of natural scenes taken on clear, foggy and hazy days. Parraga et al. [8] provide a hyperspectral dataset of 29 natural scenes. The MIT city scanning project [9] provides a set of 10000 geo-referenced calibrated images acquired over a wide area of the MIT campus. These databases, however, do not cover the complete appearance variability (due to all outdoor illumination and weather conditions) in any one particular scene. Finally, web-cams [10] used for surveillance capture images regularly over long periods of time. However, they are usually low quality, non-calibrated, not tagged with ground truth data and focus only on activity in the scene. Note that the references we have provided for various databases are by no means complete. We refer the reader to [11] for a more comprehensive listing. In this paper, we present a set of very high quality registered images of an outdoor scene, captured regularly for a period of 5 months. The viewpoint (or sensor) and the scene are fixed over time. Such a dataset is a comprehensive collection of images under a wide variety of seasons, weather and illumination conditions. This database serves a dual purpose; it provides an extensive testbed for the evaluation of existing appearance models, and at the same time can provide insight needed to develop new appearance models. To our knowledge, this is the first effort to collect such data in a principled manner, for an extended time period. The data collection is ongoing and we plan to acquire images for one year. We begin by describing the image acquisition method, the sensor calibration procedures, and the ground truth data collected with each image. Next, we illustrate the various factors that effect scene appearance using images from our database captured over 5 months. We demonstrate thorough evaluation of an existing model for outdoor weather analysis, using the image database.